Method and apparatus for imaging processing

ABSTRACT

A method and apparatus for processing an image is described. A first area having a first value is selected. A second value for a second area surrounding the first area is determined. The first and second values are compared with a predetermined threshold. The first value is transformed according to the comparison.

FIELD OF THE INVENTION

The invention relates to image processing in general. More particularly,the invention relates to a method and apparatus for increasing detailsin an image while maintaining brightness and color constancy.

BACKGROUND OF THE INVENTION

The superior quality provided by digital technologies has created a hugedemand for digital products in general. Part of this digital revolutionis the increased popularity of digital images. It is now possible to usea digital camera to capture an image and reproduce the image on somesort of display media using a personal computer (PC) monitor or highresolution printer. It has even become common practice to incorporatedigital images into “web pages” available over a network such as theInternet and World Wide Web, or to send a digital image to another PCvia electronic mail.

At the heart of this digital image revolution are image processingsystems. These systems process the captured digital image to enhance theclarity and details of the image using sophisticated image processingalgorithms. The use of these algorithms result in images that aresubstantially more accurate and detailed than previously achieved usingolder analog methods.

There remains, however, a substantial difference between how an image isperceived by a person and an image captured and reproduced on a displaymedium. Despite the improvements gained by digital image processingsystems, these systems are still incapable of reproducing an image withthe same level of detail, color constancy, and lightness of an actualscene as the eye, brain and nervous system of a human. This is due inpart because the human nervous system has a greater dynamic rangecompression than is available on current digital systems. Dynamic rangecompression refers to the ability to distinguish varying levels oflight. The human eye has a dynamic range compression of approximately1000:1, which means that the human eye can distinguish approximately1000 levels of light variations. By way of contrast, digital imagesystems typically use only eight bits per pixel which allows for adynamic range compression of only 255:1. As a result, a digital imagereproduced as a photograph would have far less detail in the darker andbrighter regions of the photograph as compared to the actual sceneperceived by a viewer.

Many techniques have been developed to compensate for this lightingdeficiency. These techniques can be separated into two broad categories:(1) power law or non-linear techniques (“non-linear techniques”); and(2) retinex techniques. Each have their respective limitations.

Non-linear techniques use a non-linear relationship to expand oneportion of the dynamic range while compressing another. These techniquesgenerally enhance details in the darker regions at the expense of detailin the brighter regions. For example, each pixel of a digital image isrepresented using eight bits, and therefore is assigned a luminancevalue somewhere in the range of 0 to 255, with 0 representing the lowestlight level and 255 the highest. If a region is comprised of pixelshaving low light values, detail is lost since there is very littlegradient between the light values for each pixel. For example, it isdifficult for a viewer to look at a picture and distinguish between apixel having a luminance value of 23 and a pixel having a luminancevalue of 25. To solve this problem, conventional image processing systemutilize non-linear techniques to increase the luminance values forneighboring pixels, thereby creating a greater degree of contrastbetween the pixels. Thus, the pixel having the luminance value of 25might be assigned a higher value such as 28 to create a greater contrastbetween it and the pixel having a value of 23.

One problem associated with these non-linear systems is that theyprovide greater distinctions between pixels regardless of what lightnessvalue a pixel may have been originally assigned. This results in thebrighter areas which already have a “washed-out” appearance to becomeeven more washed-out. Although these techniques result in greater detailin the darker regions, they do so at the expense of the brighter areasof the digital image. Further, these methods suffer from contours andabrupt boundaries.

Retinex techniques are based on the work of Dr. Edwin H. Land. Thefundamentals of Mr. Land's retinex techniques as well as severalvariations are described in a paper by A. Moore titled “SpatialFiltering in Tone Reproduction and Vision,” PhD Thesis, CaliforniaInstitute of Technology, Pasadena, Calif., 1992, and D. J. Jobson, etal. in a paper titled “A Multi-Scale Retinex for Bridging the GapBetween Color Images and the Human Observation of Scenes,” IEEETransactions on Image Processing: Special Issue on Color Processing,July 1997. In essence, these techniques increase or decrease theluminance value for a pixel based on the luminance values of surroundingpixels. These techniques are particularly useful for enhancingboundaries between lighter and darker regions of an image. Thesetechniques, however, are unsatisfactory for a number of reasons. Forexample, the Moore reference describes a technique that grays out largeuniform zones in the image. With respect to the Jobson reference, thedescribed technique allows a shift in color in some images and iscomputational intensive.

In view of the foregoing, it can be appreciated that a substantial needexists for an image processing method and apparatus that solves theabove-discussed problems.

SUMMARY OF THE INVENTION

One embodiment of the invention comprises a method and apparatus forprocessing an image. A first area having a first value is selected. Asecond value for a second area surrounding the first area is determined.The first and second values are compared with a predetermined threshold.The first value is transformed according to the comparison.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block flow diagram of an image processing system suitablefor practicing one embodiment of the present invention.

FIG. 2 is a block diagram of an image enhancer in accordance with oneembodiment of the invention.

FIG. 3 is a block flow diagram of the steps performed by an imageenhancer in accordance with one embodiment of the invention.

DETAILED DESCRIPTION

One embodiment of the invention enhances details for a digital imagewhile preserving brightness and color constancy. At a given point (e.g.,pixel) in the image, a determination is made as to whether thesurrounding pixels (“the surround”) constitute a dark or light region ofthe image. If the surround is a dark region, the inverse of the surroundis used to enhance the center of the surround. If the surround happensto be a light region, the center pixel is scaled with a linear transformin such a way that the inverse transform for the dark region and thelinear transform for the light region meet at a common gray level.

Referring now in detail to the drawings wherein like parts aredesignated by like reference numerals throughout, there is illustratedin FIG. 1 a block flow diagram of an image processing system suitablefor practicing one embodiment of the present invention. An image iscaptured and digitized according to techniques well-known in the art atstep 100. The digital image is represented by discrete areas referred toas pixels.

The digitized image is transformed to YUV space at step 102, with Yrepresenting luminance, U a first color, and V a second color. In thisembodiment of the invention, the transformation is made in accordancewith International Telecommunications Union-Radio (ITU-R) Rec. 601-2titled “Encoding Parameters of Digital Television for Studios,” 1990.Each pixel is assigned a YUV value. The Y value controls the brightnessfor that particular pixel. Current systems typically utilize eight bitsto represent the Y value due to bandwidth efficiency and memory designconsiderations. Therefore, conventional image processing systems assigneach pixel a Y value somewhere in the range of 0 to 235, with 0representing the darkest luminance and 235 representing the brightestluminance.

At step 104 image enhancement techniques are used to emphasize andsharpen image features for display. Enhancement methods operate in thespatial domain by manipulating the pixel data or in the frequency domainby modifying the spectral components. This embodiment of the inventionoperates in the spatial domain, and more particularly, applies atransform only on the luminance value Y. This embodiment of theinvention enhances the details in the darker regions of the digitallyrecorded images without washing out the details at the brighter endsthereby making the digitally recorded images more realistic with respectto an actual viewer. Further, this embodiment of the invention furtherimproves upon graying out of large uniform zones of color as occursusing conventional techniques, and also does not show color shift sinceit only operates on the Y plane. In addition, this embodiment of theinvention is computational fast since it operates on the Y plane. Theenhanced image is then outputted at step 106.

FIG. 2 is a block diagram of an image enhancer in accordance with oneembodiment of the invention. FIG. 2 shows image enhancer 200 comprisingprocessor 202 and a memory 204. Memory 204 includes computer programinstructions 206 and data 208 for implementing the main functionalityfor image enhancer 200.

The overall functioning of image enhancer 200 is controlled by processor202, which operates under the control of executed computer programinstructions that are stored in memory 204. Memory 204 comprises acomputer readable storage device. For example, memory 204 may be anytype computer readable storage device, such as random access memory(RAM), read only memory (ROM), programmable read only memory (PROM),erasable programmable read only memory (EPROM), electronically erasableprogrammable read only memory (EEPROM), magnetic storage media (i.e., amagnetic disk), optical storage media (i.e., a CD-ROM or Digital VideoDisc), and so forth. Further, image enhancer 200 may contain variouscombinations of machine readable storage devices that are accessible byprocessor 202 and which are capable of storing a combination of computerprogram instructions and data.

Processor 202 includes any processor of sufficient processing power toperform the functionality found in image enhancer 200. Examples ofprocessors suitable to practice the invention includes such processorsas the Pentium®, Pentium® Pro, and Pentium® II microprocessors made byIntel Corporation.

Computer program instructions 206 and data 208 implement the mainfunctionality for image enhancer 200. Although image enhancer 200 isdescribed in terms of software in this embodiment of the invention, itcan be appreciated that the functionality of image enhancer 200 may beimplemented in hardware, software, or a combination of hardware andsoftware, using well-known signal processing techniques.

FIG. 3 is a block flow diagram of the steps performed by an imageenhancer in accordance with one embodiment of the invention. At step300, image enhancer 200 selects a first area or pixel having a luminancevalue Y, and computes a surround value S for a second area of pixelssurrounding the first pixel. The surround S is computed using thefollowing equation:${S\left( {x,y} \right)} = {\sum\limits_{q = {y - c}}^{y + c}{\sum\limits_{p = {x - c}}^{x + c}{{Y^{\prime}\left( {p,q} \right)}*K\quad ^{{- {({{p*p} + {q*q}})}}/{({c*{c/4}})}}}}}$

The surround calculation uses the modified neighbor value Y′(p,q) asdefined above, instead of the original neighbor value Y′(p,q). Theconstants alpha and beta are percentage values which control how muchexcursion in gray scale value is allowed for the neighbors I (p, q) fromthe gray scale value of the center Y(x, y) as allowable. Advantageouspercentage values for alpha and beta are 50 and 150, respectively, for aneighborhood of c=15 pixels.

Once the surround S for a given pixel is computed, both Y and S for thegiven pixel are compared with a predetermined threshold at step 302. Yis then transformed from its initial value (e.g., a value between 0 and235) to another value based on the results of the comparison. Thetransformation occurs as follows: if both Y and S are greater than thepredetermined threshold at step 302, then Y is multiplied by thepredetermined threshold at step 304; if both Y and S are not greaterthan the predetermined threshold at step 302, then Y is multiplied bythe predetermined threshold minus S at step 306. Steps 302, 304 and 306can be mathematically represented as Y′(x, y)=Y(x, y) * (L/2), if Y>L/2and S>L/2, and otherwise as Y′(x, y)=Y(x, y) [* L-S(x, y)], whereL=(Maximum+Minimum) value of the Y scale for a given image. At step 308image enhancer 200 tests whether all pixels for the given image havebeen transformed, and if not passes control to step 300.

Multiplying a pixel with the inverse of its surround offers a superiornon-linearity that has the effect of enhancing details at the dark endas the inverse makes dark values bright and bright values dark. Theinverse surround enhancement reaches a peak value when the center pixeland its surround are equal to L/2, with L representing the maximum valueplus the minimum value of the Y scale for a given image. Thus, in thisembodiment of the invention the predetermined threshold is L/2.

The result of the inverse non-linearity enhances the bright end withoutbringing in washed-out effects that are introduced by conventionaltechniques. This embodiment of the invention offers added value forunder exposed images in particular. An under exposed image can bedetected and applied in real time by devising heuristics based on thelightness channel histogram.

This embodiment of the invention can be better understood in view of thefollowing example. Table 1 reproduced below demonstrates values for aguassion mask, pixel luminance values, a surround for each value, theoutput of the transformation equations, and rescaled pixel luminancevalues:

TABLE 1 Gaussian mask: mask = 15, gconst = 15.000000 0.000335 0.0005620.000908 0.001416 0.002131 0.000562 0.00941  0.001520 0.002371 0.0035690.000908 0.001520 0.002457 0.003832 0.005767 0.001416 0.002371 0.0038320.005976 0.008995 0.002131 0.003569 0.005767 0.008995 0.013538 EquationPixel Orig Surround Output Rescaled to  25,148 37 34.93  7994.53  72150,199 69 63.58 12931.90 136 154,242 56 47.19 11413.49 116 190,214 9089.28 14554.77 157 194,124 38 37.86  8099.16  73 (Rescaled with Max:20629.95 Min: 3709.88 to 235 and 16 respectively)

It should be noted that the above values are for exemplary purposesonly.

Although various embodiments are specifically illustrated and describedherein, it will be appreciated that modifications and variations of thepresent invention are covered by the above teachings and within thepurview of the appended claims without departing from the spirit andintended scope of the invention. For example, although a predeterminedthreshold for one embodiment of the invention is L/2, it car beappreciated that the predetermined threshold can be modified in view ofspecific design goals and still fall within the scope of the invention.Further, although the functionality for the image enhancer in thedescribed embodiments is performed in software, it can be appreciatedthat the same functionality can be implemented in hardware using adigital signal processor, application specific integrated circuit, andso forth, and still fall within the scope of the invention.

What is claimed is:
 1. A method for processing an image, comprising thesteps of: selecting a first area having a first value; determining asecond value for a second area surrounding said first area; comparingsaid first and second values with a predetermined threshold; andtransforming said first value based upon said comparison by transformingsaid first value using said predetermined threshold if both said firstand second values are above said predetermined threshold, otherwiseusing an inverse of said second value.
 2. The method of claim 1, whereinsaid first value comprises a luminance value, said second valuecomprises a surround value, and said predetermined threshold comprises agray scale value.
 3. The method of claim 1, wherein said predeterminedthreshold is represented by L/2, where L is represents a maximum plusminimum value for a luminance scale.
 4. The method of claim 1, whereinsaid step of determining determines said second value according to theformula:${{S\left( {x,y} \right)} = {\sum\limits_{q = {y - c}}^{y + c}{\sum\limits_{p = {x - c}}^{x + c}{{Y^{\prime}\left( {p,q} \right)}*K\quad ^{{- {({{p*p} + {q*q}})}}/{({c*{c/4}})}}}}}},$

where c is the gaussian spread, Y′(p, q) is defined as Y′(p, q)=Y(p, q)if αY(x,y)<=Y(p, q)<=βY(x, y), and Y′(p, q)=Y(x, y) otherwise, andconstants α and β are percentage values and K is selected such that:∫∫K  ^(−(p * p + q * q)/(c * c/4))xy =
 1.


5. An apparatus for processing an image, said image comprising aplurality of pixels, said apparatus comprising: an image transformer,said image transformer receives the image and assigns a first luminancevalue for each pixel in the image; an image enhancer coupled to saidimage transformer, said image enhancer receives the image and transformssaid first luminance value for at least one pixel to a second luminancevalue based on a set of surrounding pixels for said at least one pixeland a predetermined value wherein said first luminance value istransformed by said predetermined threshold if both said first andsecond values are above said predetermined threshold, otherwise saidfirst luminance value is transformed using an inverse of said secondvalue.
 6. A machine readable medium whose contents cause a computersystem to perform image processing by performing the steps of: selectinga first area having a first value; determining a second value for asecond area surrounding said first area; comparing said first and secondvalues with a predetermined threshold; and transforming said first valuebased upon said comparison by transforming said first value using saidpredetermined threshold if both said first and second values are abovesaid predetermined threshold, otherwise using an inverse of said secondvalue.
 7. The machine readable medium of claim 6, wherein said firstvalue is a luminance value, said second value is a surround value, andsaid predetermined threshold is a gray scale value.
 8. The machinereadable medium of claim 6, wherein said predetermined threshold isrepresented by L/2, where L represents a maximum plus minimum value fora luminance scale.
 9. The computer readable medium of claim 8, whereinsaid step of determining determines said second value according to theformula:${{S\left( {x,y} \right)} = {\sum\limits_{q = {y - c}}^{y + c}{\sum\limits_{p = {x - c}}^{x + c}{{Y^{\prime}\left( {p,q} \right)}*K\quad ^{{- {({{p*p} + {q*q}})}}/{({c*{c/4}})}}}}}},$

where c is the gaussian spread, Y′(p, q) is defined as Y′(p, q)=Y(p, q)if αY(x,y)<=Y(p, q)<=βY(x, y), and Y′(p, q)=Y(x, y) otherwise, andconstants α and β are percentage values and K is selected such that:∫∫K  ^(−(p * p + q * q)/(c * c/4))xy =
 1.


10. A method for processing a digital image having pixels, comprisingthe steps of: capturing and digitizing said digital image; selecting afirst area of pixels having a first luminance value; determining asecond luminance value for a second area of pixels surrounding saidfirst area; comparing said first and second luminance values with apredetermined threshold; and enhancing said first luminance value basedupon said comparison by transforming said first value using saidpredetermined threshold if both said first and second values are abovesaid predetermined threshold, otherwise using an inverse of said secondvalue.
 11. The method of claim 10, wherein said determining stepincludes computing a surround value for said second area of pixels.